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Abstract 

Models  of  category  learning  can  take  two  different  approaches 
to  representing  the  relationship  between  objects  and  cate¬ 
gories.  The  generative  approach  solves  the  categorization 
problem  by  building  a  probabilistic  model  of  each  category'  and 
using  Bayes’  rule  to  infer  category  labels.  In  contrast,  the  dis¬ 
criminative  approach  directly  learns  a  mapping  between  inputs 
and  category  labels.  With  this  distinction  in  mind,  we  revisit  a 
previously  studied  categorization  experiment  that  showed  peo¬ 
ple  are  biased  towards  categorizing  objects  into  a  category  with 
higher  variability.  Modelling  results  predict  that  generative 
learners  should  be  more  greatly  affected  by  category  variabil¬ 
ity  than  discriminative  learners.  We  show  that  humans  can 
be  prompted  to  adopt  either  a  generative  or  discriminative  ap¬ 
proach  to  learning  the  same  input,  resulting  in  the  predicted 
effect  on  use  of  category  variability. 
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Introduction 

Categories  can  be  learned  using  a  variety  of  approaches.  Here 
we  examine  two  distinct  approaches  that  humans  can  use 
to  learn  categories:  generative  and  discriminative  learning. 
While  relatively  unexplored  in  human  categorization,  this  dis¬ 
tinction  has  been  widely  studied  in  machine  learning  (e.g.,  Ng 
&  Jordan,  2001).  The  distinction  comes  down  to  whether  the 
ability  to  categonze  objects  is  the  result  of  estimating  a  distri¬ 
bution  for  each  category,  or  learning  a  mapping  from  objects 
to  categories.  Both  of  these  strategies  can  be  used  in  learning 
real  life  categories.  For  example,  you  could  learn  the  food 
preferences  of  a  friend  by  observing  the  foods  he  eats  and 
trying  to  infer  a  probability  distribution,  or  by  recording  his 
affective  responses  to  different  kinds  of  foods  and  trying  to 
identify  which  factors  lead  to  positive  or  negative  reactions. 

More  formally,  generative  and  discriminative  models  rep¬ 
resent  two  distinct  strategies  for  estimating  the  probability 
that  a  particular  object  belongs  to  a  category.  Generative 
learners  solve  this  problem  by  building  a  probabilistic  model 
of  each  category,  and  then  using  Bayes’  rule  to  identify  which 
category  was  most  likely  to  have  generated  the  object.  Dis¬ 
criminative  learners  estimate  the  probability  distribution  over 
category  labels  given  objects  directly.  These  different  strate¬ 
gies  have  implications  for  the  performance  of  these  mod¬ 
els.  Theoretical  and  empirical  analyses  have  shown  that  gen¬ 
erative  and  discriminative  models  differ  in  their  generaliza¬ 
tion  behavior,  as  well  as  the  speed  and  accuracy  of  learning 
(Efron,  1975;  Ng  &  Jordan,  2001 ;  Xue  &  Titterington,  2008). 

While  the  generative/discriminative  distinction  has  been 
studied  extensively  in  machine  learning  and  statistics,  it  has 
been  little  examined  in  human  behavior.  A  recent  study  has 


shown  humans  can  adopt  these  two  different  strategies  while 
learning  an  artificial  language  (Hsu  &  Griffiths,  2009).  In  this 
paper,  we  explore  whether  people  can  adopt  these  two  strate¬ 
gies  in  category  learning. 

The  paper  will  be  presented  as  follows.  First  we  will  pro¬ 
vide  an  overview  of  generative  and  discriminative  categoriza¬ 
tion  models.  Second,  we  will  review  related  work  from  the 
existing  human  categorization  literature.  Third,  we  will  re¬ 
visit  a  previously  studied  paradigm  that  showed  people  are 
sensitive  to  category  variability,  being  more  likely  to  assign 
an  object  equidistant  from  the  mean  of  two  categories  to  the 
category  with  higher  variance  (Stewart  &  Chater,  2002;  Co¬ 
hen,  Nosofsky,  &  Zaki,  2001;  Rips,  1989;  Smith  &  Sloman, 
1994).  Modelling  results  show  that  a  generative  model  ex¬ 
hibits  greater  sensitivity  to  category  variability  than  a  dis¬ 
criminative  model.  We  use  this  analysis  as  the  basis  for 
an  empirical  investigation  of  whether  human  learners  can  be 
prompted  to  take  these  two  distinct  learning  approaches.  Our 
results  support  the  idea  that  humans  adopt  generative  and  dis¬ 
criminative  approaches  when  appropriate.  This  provides  new 
insight  into  the  factors  affecting  human  category  learning. 

Generative  and  discriminative  models 

Rational  models  of  categorization  identify  the  underlying 
problem  as  one  of  estimating  the  probability  of  a  given  ob¬ 
ject  x  belonging  to  a  category  c,  as  expressed  by  the  distri¬ 
bution  p(cjjc).  The  difference  between  generative  and  dis¬ 
criminative  approaches  to  categorization  comes  down  to  how 
this  probability  distribution  is  estimated.  Generative  models 
build  a  probabilistic  model  of  the  input  by  learning  the  prob¬ 
ability  that  an  object  x  is  generated  given  that  the  category  is 
c,  p(x\c)y  and  then  solving  the  categorization  problem  by  ap¬ 
plying  Bayes’  rule.  Discriminative  models  estimate  p{c\x)  di¬ 
rectly.  Generative  models  thus  assume  that  observed  objects 
are  sampled  in  a  way  that  reflects  p(x|c),  while  discriminative 
models  do  not  make  any  assumptions  about  the  distribution 
from  which  the  input  is  sampled.  These  two  approaches  to 
categorization  are  illustrated  schematically  in  Figure  l. 

Comparison  of  generative  and  discriminative  approaches 
to  category  learning  has  been  done  in  the  machine  learn¬ 
ing  and  statistics  literature,  where  the  classic  generative- 
discriminative  pair  being  compared  is  usually  (generative) 
naive  Bayes  vs.  (discriminative)  logistic  regression  (Efron, 
1975;  Ng  &  Jordan,  2001;  Xue  &  Titterington,  2008). 
Under  certain  conditions,  these  two  models  are  identical 
in  the  asymptotic  form  of  the  function  p(c\x)  that  they 
produce,  differing  only  in  how  that  function  is  estimated. 
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Figure  1:  Generative  and  discriminative  models.  Generative 
models  aim  to  estimate  the  probability  distribution  over  the 
input  given  the  category  label.  Discriminative  models  find  a 
direct  mapping  between  inputs  and  category  labels. 

Such  generative-discriminative  pairs  can  thus  be  used  to  ex¬ 
plore  the  consequences  of  adopting  these  different  strategies 
through  mathematical  analysis  and  simulations.  For  exam¬ 
ple,  if  the  training  data  consist  of  two  normally  distributed 
samples,  generative  models  learn  categories  more  quickly 
(Efron,  1975;  Ng  &  Jordan,  2001).  However,  when  the  train¬ 
ing  data  come  from  other  distributions,  discriminative  models 
are  asymptotically  more  accurate  (Xue  &  Titterington,  2008), 
though  in  some  cases  generative  models  may  perform  bet¬ 
ter  initially  and  arrive  at  their  (higher)  asymptotic  error  more 
quickly  (Ng  &  Jordan,  2001). 

Summary  of  related  work 

Previous  models  of  categorization  have  used  both  genera¬ 
tive  and  discriminative  strategies,  without  necessarily  rec¬ 
ognizing  that  the  significance  of  the  distinction.The  com¬ 
monly  cited  prototype  and  exemplar  models  can  be  applied 
both  generatively  and  discriminatively.  Prototypes  and  ex¬ 
emplars  are  psychological  models  of  category  representation 
whereas  discriminative  and  generative  are  statistical  models 
of  learning.  Thus,  prototype  and  exemplar  models  can  be 
used  under  either  approach,  depending  on  how  learning  takes 
place.  For  instance,  ALCOVE  (Kruschke,  1992)  is  an  ex¬ 
emplar  model  akin  to  discriminative  kernel  methods.  SUS¬ 
TAIN  (Love,  Medin,  &  Gureckis,  2004)  is  a  discriminative 
model  that  chooses  between  exemplar  and  prototype  repre¬ 
sentations.  Decision  bounds  (Maddox  &  Ashby,  1993)  can  be 
either  discriminative  or  generative  depending  on  how  model 
parameters  are  estimated.  While  rational  models  of  catego¬ 
rization  can  adopt  either  approach,  the  ones  proposed  so  far 
have  taken  a  generative  approach  (e.g.,  J.  R.  Anderson,  1990; 
Griffiths,  Canini,  Sanborn,  &  Navarro,  2007).  These  genera¬ 
tive  categorization  models  span  the  range  between  exemplar 
and  prototype  representations.  At  the  extremes,  generative 
prototype  models  estimate  parameters  of  category  distribu¬ 
tions  (usually  a  Gaussian  with  a  mean  and  variance)  and  gen¬ 


erative  exemplar  models  estimate  category  distributions  using 
kernel  density  estimation  (Ashby  &  Alfonso-Reese,  1995). 

Despite  the  prevalence  of  human  categorization  models 
with  both  discriminative  and  generative  approaches,  most  ex¬ 
perimental  paradigms  seem  more  consistent  with  discrimi¬ 
native  learning:  stimuli  are  presented,  participants  guess  the 
category  and  feedback  is  given.  However,  a  few  exceptions 
this  can  be  seen  in  previous  work  on  classification  vs.  in¬ 
ference  learning,  and  observational  vs.  feedback  learning. 
While  not  explicitly  mentioned  in  previous  work,  both  of 
these  paradigms  are  potentially  related  to  our  discriminative 
vs.  generative  distinction. 

Classification  vs.  inference  learning 

Another  line  of  experiments  has  shown  that  human  category 
learning  can  also  be  influenced  by  using  different  tasks  to 
teach  people  about  the  relationship  between  categories  and 
features.  The  effect  of  using  these  two  different  tasks  is  sim¬ 
ilar  to  that  of  changing  the  direction  of  a  learned  causal  re¬ 
lationship.  (A.  L.  Anderson,  Ross,  &  Chin-Parker,  2002; 
Markman  &  Ross,  2003;  Ross  &  Murphy,  1996).  In  these 
experiments,  all  participants  were  presented  with  exacdy  the 
same  training  stimuli,  consisting  of  the  features  and  category 
membership  of  a  set  of  objects.  In  one  condition,  learning 
took  place  via  through  classification:  Participants  were  pro¬ 
vided  with  the  values  for  (some  of)  the  features  of  an  object 
asked  to  predict  category  membership.  In  the  other  condi¬ 
tion,  learning  was  based  on  making  a  predictive  inference : 
The  category  membership  and/or  values  of  some  of  the  fea¬ 
tures  were  provided  and  participants  were  asked  to  predict 
the  value  of  another  feature.  Because  participants  in  both 
conditions  were  given  feedback,  they  were  both  ultimately 
provided  with  exactly  the  same  information  about  categories 
and  features.  However,  learning  results  differed  in  terms  of 
performance  accuracy  and  generalizations  made.  For  exam¬ 
ple,  inference  learners  performed  better  than  classification 
learners  on  single-feature  classification  tasks  but  more  poorly 
when  all  of  the  features  were  provided  (A.  L.  Anderson  et 
al.,  2002).  While  this  study  was  not  motivated  by  generative 
and  discriminative  learning,  people  may  have  adopted  these 
different  strategies  in  the  different  conditions:  Classification 
learning  can  be  done  using  a  discriminative  model,  while  in¬ 
ference  learning  requires  a  generative  model. 

Observation  vs.  feedback  training 

Another  study,  by  Ashby,  Maddox,  and  Bohill  (2002),  has 
also  examined  how  learning  of  the  exact  same  input  was  af¬ 
fected  by  presentation  style.  Here  they  compared  what  they 
called  feedback  training  (where  the  category  label  appears  af¬ 
ter  the  object)  with  observation  training  (where  the  category 
label  appears  before  the  object).  Their  results  showed  that 
participants  in  the  feedback  condition  performed  significantly 
better  than  those  in  the  observation  condition  for  information- 
integration  categories,  where  category  membership  could  not 
be  expressed  in  terms  of  a  rule  using  a  single  feature.  These 
two  forms  of  training  might  encourage  learners  to  adopt  gen- 


erative  and  discriminative  strategies.  Feedback  training  gives 
an  error  signal  that  can  be  used  to  adapt  a  discriminative 
model.  Observation  training  is  more  relevant  for  learning  ob¬ 
ject  features  based  on  the  category  label,  which  is  the  gener¬ 
ative  approach. 

Summary 

Generative  and  discriminative  models  use  different  ap¬ 
proaches  to  solve  the  problem  of  categorizing  objects.  Ex¬ 
isting  models  of  human  category  learning  differ  in  which  of 
these  approaches  they  use.  Previous  work  has  not  explored 
whether  people  are  able  to  switch  the  approach  they  take  in 
learning  categories,  although  the  effects  of  different  training 
regimes  that  might  encourage  one  approach  over  the  other 
have  been  investigated.  In  the  remainder  of  the  paper,  we  ex¬ 
plicitly  test  whether  people  can  adopt  these  two  approaches 
to  learning  categories,  using  a  phenomenon  that  is  diagnostic 
for  one  generative-discriminative  pair  of  models. 

Differential  use  of  category  variability 

Several  experiments  have  shown  an  effect  of  category  vari¬ 
ability  on  human  categorization  judgments.  In  these  experi¬ 
ments,  the  stimuli  belong  to  one  of  two  categories  with  dif¬ 
ferent  means  and  variances.  The  key  question  is  how  stimuli 
with  features  lying  (perceptually)  in  between  the  two  cate¬ 
gories  are  categorized.  The  results  of  these  experiments  all 
showed  that  there  was  a  bias  towards  categorizing  stimuli  into 
the  high-variance  category  (Stewart  &  Chater,  2002;  Cohen 
et  al.,  2001;  Rips,  1989;  Smith  &  Sloman,  1994).  Here  we 
propose  that  the  degree  of  preference  for  the  high  variance 
category  may  be  affected  by  whether  the  learner  is  adopting 
a  generative  or  discriminative  approach. 

Intuitively,  we  expect  category  variability  to  have  a  greater 
effect  on  generative  learners  because  estimating  /?(x|c)  for 
each  category  requires  being  sensitive  to  the  variance  of  that 
category.  In  contrast,  one  need  not  consider  the  variance  of 
the  stimuli  in  simply  learning  a  function  from  x  to  c,  p(c|x). 
Indeed  many  discriminative  models  used  in  machine  learn¬ 
ing,  such  as  support  vector  machines  (Scholkopf  &  Smola, 
2002),  focus  just  on  the  location  of  the  most  extreme  mem¬ 
bers  of  each  category.  We  are  not  claiming  that  all  generative 
models  are  sensitive  to  category  variance,  or  that  all  discrimi¬ 
native  models  are  insensitive,  but  that  these  approaches  differ 
in  the  extent  to  which  they  are  sensitive  to  this  property  of  the 
stimuli.  To  illustrate  this,  we  will  explore  the  predictions  of 
one  generative-discriminative  pair  of  models. 

We  follow  previous  work  exploring  the  difference  between 
generative  and  discriminative  models  (e.g.,  Ng  &  Jordan, 
2001)  and  focus  on  the  generative -discriminative  pair  of  naive 
Bayes  and  logistic  regression.  Since  we  will  focus  on  contin¬ 
uous  stimuli,  we  assume  a  Gaussian  generative  model,  with 

p(x\c  =  i)  =  N(ph  f)  (1) 

where  pt  and  *  are  the  mean  and  variance  of  the  ith  category 
with  i*  €  {1,2}.  The  parameters  ju,  and  ,•  can  be  estimated 


by  maximizing  the  likelihood  £ysilogp(x;|cj,/i,  ),  where 
cj  and  xj  are  the  category  membership  and  features  of  the 
jrth  stimulus  respectively.  The  probability  a  novel  stimulus 
belongs  to  a  category,  p(c\x),  is  then  computed  by  applying 
Bayes’  rule,  with  the  prior  probability  of  each  category  being 
proportional  to  the  number  of  observed  stimuli  from  that  cat¬ 
egory.  The  naive  Bayes  model  is  similar  to  the  Gaussian  deci¬ 
sion  bound  model  used  in  Normal  general  recognition  theory 
(Stewart  &  Chater,  2002;  Maddox  &  Ashby,  1993). 

The  discriminative  model  uses  logistic  regression  to  esti¬ 
mate  p{c\x)  directly,  with 

p(c  =  l|jc,w,&)  =  1/(1  +  exp{— wrx) -b})  (2) 

where  w  and  b  are  the  parameters  of  the  model  and  x  is  a  vec¬ 
tor  of  feature  values.  The  parameters  w  and  b  are  estimated  by 
maximizing  the  log  likelihood  Yj=  i  l°gp(fj|xy,  w,6).  In  gen¬ 
eral,  w  and  b  are  vectors  of  length  equal  to  the  number  of  stim¬ 
ulus  features.  However,  we  will  be  using  one-dimensional 
stimuli  (xj  is  scalar),  so  w  and  b  will  be  scalars  in  our  case. 
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Figure  2:  Stimuli  used  in  the  experiment.  Category  A  and  B 
were  the  low  and  high  variance  categories  respectively 

To  examine  the  predictions  of  these  models,  we  used  stim¬ 
uli  based  largely  on  those  of  Cohen  et  al.  (2001).  Stimuli 
consisted  of  vertical  lines  of  varying  lengths.  Training  stim¬ 
uli  belonged  to  one  of  two  categories,  A  and  B.  Category  A 
is  the  low  variance  category.  Category  A  contained  lines  of 
length  110, 120, 130, 140  and  150  pixels.  Category  B  was  the 
high  variance  category.  Category  B  contained  lines  of  length 
300,  375,  450,  525  and  600  pixels.  All  stimuli  were  equally 
likely  within  each  category  (categories  had  a  flat  distribution 
of  stimuli).  We  also  included  novel  transfer  stimuli  in  the  test 


stimuli.  There  were  eight  transfer  stimuli,  equally  spaced  be¬ 
tween  the  highest  value  of  A  and  the  lowest  value  of  B  (see 
Figure  2).  A  range  of  intermediate  transfer  stimuli  were  used 
in  case  the  middle  stimulus  in  psychological  space  differed 
from  the  numerical  middle  stimulus.  The  precise  location  of 
the  middle  stimulus  is  not  important  for  our  purposes,  as  the 
difference  in  results  between  generative  and  discriminative 
models  is  the  question  of  interest. 

Model  categorization  results 


Figure  3:  Generative  and  discriminative  model  predictions 
for  the  probability  of  categorization  stimuli  into  the  high  vari¬ 
ance  category.  The  model  predictions  are  that  a  generative 
learner  is  more  likely  to  categorize  in  between  stimuli  in  the 
high  variance  category 

We  trained  a  generative  naive  Bayes  model  and  discrimi¬ 
native  logistic  regression  model  on  all  labeled  examples  from 
category  A  and  B.  Our  naive  Bayes  model  had  uniform  cate¬ 
gory  priors,  i.e.  both  categories  were  assumed  to  be  equally 
likely.  Parameters  for  both  models  were  fit  using  maximum 
likelihood  estimation.  To  compare  the  outcomes  of  the  two 
models,  we  analysed  categorization  predictions  for  our  trans¬ 
fer  stimuli  using  these  generative  and  discriminative  models. 
The  generative  model  predicts  intermediate  transfer  stimuli 
will  be  classified  to  the  high-variance  category  more  often 
than  the  discriminative  model  (see  Figure  3).This  is  because 
it  is  more  likely  that  intermediate  stimuli  are  extreme  val¬ 
ues  from  the  high-variance  category  than  the  low-variance 
category.  These  results  illustrate  that  sensitivity  to  category 
variability  may  be  a  diagnostic  indicator  of  whether  learn¬ 
ers  are  using  a  generative  or  a  discriminative  strategy.  In  the 
next  section  we  present  an  experiment  that  uses  this  indicator 
to  determine  whether  human  learners  switch  between  these 
strategies  depending  on  the  way  in  which  a  categorization 
task  is  presented. 

Human  generative  and  discriminative  learning 

Method 

Participants  We  collected  data  from  24  participants  (12  in 
each  condition).  Participants  wereundergraduates  at  the  Uni¬ 
versity  of  California,  Berkeley  and  received  course  credit. 


Stimuli  Stimuli  was  the  same  training  and  transfer  stimuli 
used  in  the  model  simulations  described  in  the  previous  sec¬ 
tion.  In  the  experiment,  these  stimuli  were  presented  as  white 
vertical  lines  in  a  black  circle. 

Procedure  While  previous  related  work  had  paradigms  that 
may  have  encouraged  discriminative  or  generative  learning 
(Ashby  et  al.,  2002;  A.  L.  Anderson  et  al.,  2002),  the  connec¬ 
tion  between  these  paradigms  and  the  distinction  was  tenta¬ 
tive.  Thus,  we  will  use  our  own  experimental  manipulation 
in  order  to  encourage  participants  to  adopt  the  distinct  ap¬ 
proaches  as  strongly  as  possible.  Participants  in  both  learn¬ 
ing  conditions  were  trained  under  the  same  randomized  se¬ 
quence  of  trials.  In  order  to  prompt  generative  or  discrimina'* 
five  learning,  the  two  conditions  differed  in  the  instructions, 
category-stimulus  presentation  order  and  question  presented 
during  testing  blocks.  Participants  in  both  conditions  were 
told  they  will  see  ‘‘signs”  from  an  alien  tribe.  Participants  in 
the  generative  condition  were  told  that  two  aliens,  one  from 
each  tribe  (A  and  B)  will  appear  and  produce  signs  from  their 
respective  tribes.  A  picture  of  two  aliens,  who  were  identi¬ 
cal  except  for  the  letter  on  their  chest,  was  shown  alongside 
the  instructions.  These  instructions  were  intended  to  make  it 
clear  that  the  observed  stimuli  were  generated  from  a  prob¬ 
ability  distribution  associated  with  the  target  category,  con¬ 
sistent  with  the  assumptions  of  a  generative  model.  Partici¬ 
pants  in  the  discriminative  condition  were  told  that  there  are 
signs  from  two  alien  tribes  and  they  would  be  shown  a  single 
alien  translator  who  can  report  which  tribe  a  sign  was  from. 
A  single  alien  was  shown  alongside  these  instructions  with  a 
question  mark  on  its  chest.  These  instructions  were  intended 
to  establish  a  situation  in  which  participants  learned  a  func¬ 
tion  from  stimuli  to  category  membership,  consistent  with  a 
discriminative  model. 

For  all  participants,  the  experiment  contained  10  blocks 
of  20  trials  (each  of  10  training  stimuli  were  shown  twice). 
Training  blocks  (odd  blocks)  were  interleaved  with  testing 
blocks  (even  blocks).  During  training  trials,  participants  were 
shown  a  black  circular  background  on  which  the  “sign”  ap¬ 
pears  as  a  white  vertical  line,  next  to  an  alien  with  either  A 
or  B  written  on  its  chest  In  the  generative  condition,  the 
alien  appeared  500  ms  before  the  sign  during  training  and  the 
alien  disappeared  between  trials  to  simulate  different  aliens 
appearing.  In  the  discriminative  condition,  the  sign  appeared 
500  ms  before  the  alien  and  the  alien  did  not  disappear  be¬ 
tween  trials  to  simulate  one  constant  alien  interpreter.  In  both 
conditions, once  both  stimulus  and  letter  had  appeared,  both 
remained  simultaneously  on  the  screen  for  1.5  s  (see  Figure 
4).  The  total  length  of  each  training  trial  was  2  s  and  there 
were  700  ms  between  each  trial. 

During  test  trials,  participants  were  shown  a  sign  (white 
vertical  line)  on  the  black  circular  background.  Participants  in 
the  generative  condition  were  asked  “Which  alien  was  more 
likely  to  have  produced  this  sign?”.  Participants  in  the  dis¬ 
criminative  condition  were  asked  “Which  alien  tribe  does  this 
sign  belong  to?”.  Stimuli  during  each  test  block  consisted  of 


every  example  stimulus  in  categories  A  and  B,  along  with  the 
eight  transfer  stimuli  that  were  equally  spaced  between  and 
highest  value  of  category  A  and  the  lowest  value  of  category 
B.  (The  highest  value  of  category  A  and  lowest  value  of  cat¬ 
egory  B  were  seen  twice  during  each  test  block  to  make  up 
the  20  trials.)  No  feedback  was  given  during  testing  in  either 
condition. 


Figure  4:  Screen  shot  of  the  experiment 


Results 

The  human  learning  results  correspond  to  the  predictions  of 
the  models:  Generative  learners  are  more  likely  to  catego¬ 
rize  transfer  stimuli  that  lie  in  between  the  two  categories  in 
the  high-variance  category  relative  to  discriminative  learners 
(see  Figure  5).  A  two-way  within-between  ANOVA  revealed 
statistically  significant  effects  of  test  stimulus  (F(9, 198)  = 
76.88,  MSE  =  0.036,  p  <  .001)  and  condition  (F(l,22)  = 
5.43,  MSE  s  0.216,  p  <  .05)  and  a  marginally  significant 
interaction  (F(9,198)  =  1.90,  MSE  =  0.036,  p  =  .054). 
Planned  comparisons  using  two-sample  t-tests  showed  statis¬ 
tically  significant  effects  of  condition  for  stimuli  216  0(22)  = 
2.57,  p  <  .05)  and  233  0(22)  =  2.46,  p  <  .05).  These  statis¬ 
tics  are  calculated  under  the  most  conservative  assumption, 
under  which  the  responses  from  each  participant  for  each 
stimulus  are  averaged  together  and  treated  as  a  single  re¬ 
sponse. 

The  “middle  stimulus’*  that  lies  midway  between  the  two 
categories  in  human  perceptual  space  (i.e.  equally  likely  to  be 
categorized  in  both  categories  in  the  discriminative  condition) 
is  of  length  around  200  pixels.  This  is  smaller  than  the  nu¬ 
merical  middle  (225  pixels).  This  is  approximately  the  same 
value  as  the  perceptual  “middle  stimulus”  that  was  found  in 
previous  work  (Cohen  et  al.,  2001).  Accounting  for  this  shift, 
the  discriminative  model  predictions  match  fairly  well  with 
the  discriminative  human  results.  The  generative  model  pre¬ 
dictions  are  significandy  shifted  to  the  left  compared  with  our 
generative  human  results,  meaning  the  generative  model  pre¬ 
dicted  an  even  stronger  tendency  to  categorize  the  in-between 
stimuli  in  the  high  variance  category.  This  difference  in  de¬ 
gree  between  model  predictions  and  human  judgments  could 
be  explained  in  many  possible  ways.  One  possibility  is  that 
perceptual  stimuli  might  follow  Weberian  compression  for 


the  larger  stimuli  (Stewart  &  Chater,  2002).  As  a  result  of  this 
compression,  the  perceptual  variability  of  the  longer  length 
lines  (which  made  up  the  high  variability  category)  may  have 
been  significantly  smaller  than  the  absolute  numerical  vari¬ 
ability  values  that  were  used  in  our  models.  If  this  were  the 
case,  a  suitable  transformation,  such  as  to  log  space,  would 
leave  our  qualitative  results  the  same,  while  resulting  in  an 
appropriately  less  strong  variability  preference  for  the  gener¬ 
ative  model.  Another  possibility  is  that  people  are  not  making 
the  Gaussian  assumption  that  was  made  by  our  model.  This  is 
plausible  as  our  stimuli  were  very  non-Gaussian.  In  this  case, 
it  is  possible  that  the  probability  of  belonging  in  the  high  vari¬ 
ance  category  under  a  Gaussian  assumption  is  greater  than  the 
probability  estimates  that  generative  participants  might  have 
made  f6r  our  actual  stimuli.  Finally,  participants  may  not  be 
behaving  fully  generatively,  or  that  the  instructions  resulted  in 
a  mixed  population  of  generative  and  discriminative  learners 
in  this  condition. 


Human  categorization  results 


Figure  5:  Probability  of  categorizing  transfer  stimuli  in  high 
variance  category  for  participants  in  the  generative  and  dis¬ 
criminative  learning  conditions.  Total  values  are  the  average 
of  all  probabilities  for  individual  stimulus  lengths. 

Discussion 

The  distinction  between  generative  and  discriminative  ap¬ 
proaches  to  categorization  has  played  an  important  role  in 
machine  learning  research,  but  has  not  previously  been  ex¬ 
plored  in  cognitive  psychology.  Our  results  show  that  people 
can  be  cued  to  take  these  two  different  approaches  to  category 
learning  through  the  way  in  which  a  categorization  task  is 
presented.  These  results  have  implications  for  understanding 
human  category  learning,  and  for  establishing  links  between 
the  communities  studying  human  and  machine  learning. 

The  finding  that  people  behave  differently  when  encour¬ 
aged  to  adopt  these  two  different  approaches  to  category 
learning  may  shed  light  on  previous  empirical  results  in 
cognitive  psychology.  For  example,  some  previous  experi¬ 
ments  have  shown  effects  that  may  be  partly  due  to  learn¬ 
ing  paradigms  that  encouraged  participants  to  adopt  genera¬ 
tive  or  discriminative  learning  approaches  (e.g.,  Ashby  et  al., 


2002).  The  generative/discriminative  distinction  also  has  po¬ 
tential  implications  for  previously  proposed  models  of  cate¬ 
gorization.  For  example,  it  seems  appropriate  that  connec- 
tionist  models  (Kruschke,  1992;  Love  et  al.,  2004)  will  best 
characterize  behavior  when  humans  adopt  a  discriminative 
learning  approach  whereas  rational  models  (J.  R.  Anderson, 
1990;  Griffiths  et  al.,  2007)  will  best  describe  behavior  when 
humans  adopt  a  generative  learning  approach.  Developing 
a  deeper  understanding  of  how  this  distinction  plays  out  in 
human  learning  may  provide  additional  insights  into  long¬ 
standing  debates  on  category  learning. 

Showing  that  people  can  adopt  both  generative  and  dis¬ 
criminative  learning  strategies  establishes  a  new  connection 
between  human  and  machine  learning.  While  many  of  the 
goals  of  machine  learning  are  inspired  by  human  capabilities 
(e.g.,  the  ability  to  recognize  and  categorize  complex  struc¬ 
tures  quickly  and  efficiently),  the  principal  issues  that  are 
topical  in  machine  and  human  learning  seldom  coincide.  By 
showing  that  a  key  distinction  long  studied  in  machine  learn¬ 
ing  research  is  also  significant  to  human  learning,  this  work 
begins  to  build  an  important  bridge  between  machine  learn¬ 
ing  and  human  learning  communities.  This  will  encourage 
collaboration  between  the  two  research  communities  where 
computational  models  of  learning  provide  insight  into  human 
learning  and  human  learning,  in  turn,  inspires  computational 
modelling.  It  also  establishes  a  way  to  know  how  advances 
in  specific  aspects  of  machine  learning,  such  as  improved  dis¬ 
criminative  models,  might  be  relevant  to  predicting  aspects  of 
human  learning. 

Identifying  the  relevance  of  the  generative/discriminative 
distinction  in  human  categorization  also  opens  up  many  new 
avenues  of  research  questions.  For  the  neuroscience  com¬ 
munity,  one  can  ask:  What  neural  mechanisms  are  imple¬ 
menting  these  two  very  different  learning  strategies?  Are  the 
neural  circuits  involved  similar  or  different?  This  research 
also  provokes  many  questions  about  learning  more  generally: 
When  does  human  learning  tend  to  be  generative  or  discrimi¬ 
native?  How  flexible  are  learners  in  alternating  between  gen¬ 
erative  and  discriminative  learning  approaches?  Can  learn¬ 
ing  approaches  be  retrospectively  altered?  (i.e.  if  input  is 
learned  with  a  discriminative  perspective  and  learners  were 
later  made  to  understand  that  the  data  was  generated  from  a 
probability  distribution,  would  they  switch  their  categoriza¬ 
tion  judgments?)  Since  much  of  human  learning  in  everyday 
life  consists  of  a  mix  of  scenarios  in  which  one  or  the  other  of 
these  strategies  is  more  appropriate,  clarifying  when  people 
use  generative  and  discriminative  approaches  will  help  us  un¬ 
derstand  differences  in  learning  among  individuals  and  across 
situations.  We  anticipate  that  exploring  these  questions  will 
result  in  improved  models  of  human  category  learning,  and 
a  tighter  coupling  between  research  on  human  and  machine 
learning. 
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